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Abstract 

We consider stochastic impulse control problems where the process is driven by a general one- 
dimensional diffusions. Impulse control problems are widely used to financial engineering/decision- 
making problems such as dividend payout problem, portfolio optimization with transaction costs, 
and inventory control. We shall show a new mathematical characterization of the value function 
as a linear function in certain transformed space. Our approach can (1) relieve us from the bur- 
den of guessing and proving the optimal strategy, (2) present a simple method to find the value 
function and the corresponding control policies, and (3) handle systematically a broader class 
of reward and cost functions than the conventional methods of quasi variational inequalities, 
especially because the existence of the finite value function can be shown in much a simpler way. 
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1 Introduction 

This paper proposes a general solution method of stochastic impulse control problems for one dimen- 
sional diffusion processes. Stochastic impulse control problems have attracted a growing interest 
of many researchers for the last two decades. Under a typical setting, the controller faces some 
underlying process and reward/cost structure. There exist continuous and instantaneous compo- 
nents of reward/cost functions. By exercising impulse controls, the controller moves the underlying 
process from one point to another. At the same time, the controller receives rewards associated 
with the instantaneous shifts of the process. Then the controller's objective is to maximize the 
total discounted expected net income. 
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Optimal Stopping for One- dimensional Diffusions. 
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The mathematical framework to these types of problems is in Bensoussan and L ions (|1984l ). 
Impulse control has been studie d wide ly in inventory control (H arriso n et al. (|1983l )). exchange 
rate p roblem (Jeanblanc-Picque (jl993 ). Mundaca and 0ksendal Jl998T) . Cadenillas and Zapatero 
(|2000l )). dividend payout problems (Jeanblac-Picque and ShiryaevJ11995|)), and portfo lio optimiza- 
tion with transaction costs (Korn (1998J), Morton and Pliska (1995)). Korn (1999) surveys the 
applications in mathematical finance. Also see Chancelier et al. (|2002l ) for a combination of optimal 
stopping and impulse control problems. In many economic and financial applications where the con- 
trolled process is described as an ltd diffusion, the solution to the problem demands a through study 
of a related Hamilton-Jacobi-Bellman equation and quasi-variational inequalities. The method of 
quasi-variational inequalities split by a guess the state space into intervention and no intervention 
(continuation) regions. One guesses the form of (a) continuation region, (b) associated optimal 
policy, and (c) the value function. Then optimality of the candidate policy must be verified. Both 
steps are often very difficult and the success depends heavily on the form of the controlled process, 
reward and cost functions. 

Alternatively, an impulse control problem can be viewed as a sequence of optimal stopping 
proble ms. T he connection between impu lse co ntrol and optimal stopping has been investigated by 
Davis (jl992l ) and 0ksendal and Sulem J2QO2I) 

among others. In this setting, the value functions 
of a sequence of optimal stopping problems converge to the value function of the impulse control 
problem under suitable conditions. 



In this paper, we utilize this connection together with a novel method of Dayanik and Karatzas (|2003l ) 
for optimal stopping problems. We use it to identify a new and useful characterization of the so- 
lution of the original impulse control problem. At the end we get rid of the sequence of optimal 
stopping problems altogether: the new characterization allow us to propose a new direct solution 
method for impulse control problems. 

In the next section, we briefly go over the solution method for optimal stopping problems of 
one-dimensional diffusions. We describe the impulse control problem and its solution in Section 3. 
Examples are presented in Section 4. Finally, extensions and concluding remarks are in Section 5. 



2 Summary of the Key Results of Optimal Stopping 

Let (O, P) be a complete probability space with a standard Brownian motion W = {Wt]t > 0} 
and consider the diffusion process X° with state pace IC1 and dynamics 

dXf = n{X?)dt + a(X?)dW t (2.1) 

for some Borel functions fi : I — > M and a : I — ► (0,oo). We emphasize here that X° is an 
uncontrolled process. We assume that I is an interval with endpoints —00 < a < b < +00, and 
that X° is regular in (a, b); in other words, X° reaches y with positive probability starting at x for 
every x and y in (a, b). We shall denote by F = {jF f } the natural filtration generated by X°. 

Let a > be a real constant and h(-) a Borel function such that K x {e~ aT h(X®)] is well-defined 
for every F-stopping time r and x £ I. Let r y be the first hitting time of y £ X by X°, and let 
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c G X be a fixed point of the state space. We set: 

jV[ e — cl ^ X < C} fl/E c [e-^l {T;c<oo} ], x<c, 

ip(x) = < <p(x) = < 

[l/E c [e~ a ^l {Tx<oo} ], x>c, (E x [e- a ^l {Tc<oo} ], x > c, 

and 

Then F(-) is continuous and strictly increasing. It should be noted that ip(-) and ip(-) consist of 
an increasing and a decreasing solution of the second-order differential equation (.A — a)u = in 
X where A is the infinitesimal generator of X°. They are linearly independent positive solutions 
and uniquely determined up to multiplication. For the complete characterizatio n of ? />(•) and </?(•) 
corresponding to various types of boundary behavior, refer to Ito and McKean ()197J). 

Let F : [c,d] — ► R be a strictly increasing function. A real valued function u is called F -concave 
on [c, d] if, for every a < I < r < b and x £ [I, r], 

,*Hr)-Hx) , ,F(x)-F(l) 

We denote by 

V(x) 4 sup E x [e- ar h(X% x £ [c, d] (2.3) 
res 

the value function of the optimal stopping problem with the reward function h(-) where the supre- 
mum is taken over the class S of all F-stopping t imes. Then we have the following results, the 
proofs of which we refer to Dayanik and Karatzas (|2003l ) . 

Proposition 2.1. The value function V(-) of ()2.3[) is the smallest nonnegative majorant of h(-) 
such that V(-)/ip(-) is F-concave on [c,d]. 

Proposition 2.2. Let W(-) be the smallest nonnegative concave majorant of H = {h/ip) o F^ 1 
on [F(c), F(d)], where J* 1_1 (-) is the inverse of the strictly increasing function F(-) in (|2.2p . Then 

V(x) = ip(x)W(F(x)) for every x G [c,d]. 

Proposition 2.3. Define 

S = {xe [c, d] : V(x) = h(x)}, and r* 4 inf{t ^ : X$ G 5}. (2.4) 
If /i(-) is continuous on [c, d], then r* is an optimal stopping rule. 



3 Impulse control problems and its solution 

Suppose that at any time t G R+ and any state x G R+, we can intervene and give the system an 
impulse Once the system gets intervened, the point moves from x to y G M+ with associated 

rewards earned. An impulse control for the system is a double sequence, 

v = (Ti, T 2 , .. ..2*. £2, ••■&••■•) 
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where < T\ < T2 < .... are an increasing sequence of F-stopping times and £1, £2-" are Ttc 
measurable random variables representing impulses exercised at the corresponding intervention 
times Ti with £, € Z for all i where Z £ R is a given set of admissible impulse values. The 
controlled process is, in general, described as follows: 

dX t = fi{X t )dt + a(X t )dW u T i ^ 1 <t<T i (3.1) 

x T% = rp^_,&) (3.2) 

with some mapping r : R x R — > R. 

In this section, we consider the absorbing boundary problem. Let be the absorbing state, 
without loss of generality, and to = inf{t : Xt = 0} the ruin time. With the absorbing state at 
0, it is natural to consider a set of problems where Z G R+ (i.e., £j = X{ — yi > for all i) and 
Xti = Xt^ — £«• (We shall comment on cases where interventions are allowed in both positive and 
negative directions in section [5j) 

With each pair (Tj,£j), we associate the interventions 

K(X Ti .,X Tt ) (3.3) 

where K(x, y) : R x R — > R is a given continuous function in the first and second argument that 
represents benefit/cost at interventions. Our result below does not depend on the specification of 
K (•). We assume that, for any point x E R, 

K(x,x)<0. (3.4) 

due to the fixed cost incurred. We consider the following performance measure with v € V, a 
collection of admissible strategies, 

r-TQ 



r{x) = w 



/"TO 

/ e- as f(X s )ds + e- aT °P+ V e- aT *K(X Ti _,X Tt ) 

JO rT7_ 



(3.5) 



Ti<ro 

where P G R_ is a constant penalty at the ruin time and / : R — > R is a continuous function, 
satisfying : 



W 



< 00. (3.6) 



poo 

/ e— \f(X s )\ds 
Jo 

Our goal is to find the optimal strategy v*{Ti, £j)j>o and the corresponding value function, 

v(x) = supJ u (x) = r\x). (3.7) 

z/gV 

Let us briefly go over our plan. In section 13.11 we shall characterize optimal intervention times 
Tj as exit times of the process X from an interval by implementing recursive optimal stopping 
scheme that eventually solves the original impulse control problem. Using the results, in section 
13.21 we consider a special case where the mapping x — > —(x) : R+ — > R+ is F-concave. We show, 
under this assumption, that the optimal intervention times Tj are characterized as exit times from 
an interval, say (0,6*) for every i. Then we characterize the value function for impulse control 
problems and present a solution method based on the characterization of the intervention times 
and value function. In section [331 we consider the general case where the F-concavity assumption 
above does not hold. 
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3.1 A sequence of optimal stopping problems 

In this subsection, we consider a recursive optimal stopping with a view to characterizing interven- 
tion times for the impulse control problems. Here we assume that no absorbing boundary exists. 
As we will see in the next subsection, the existence of an absorbing state is easily incorporated. 
Hence by using the same v(x), we consider the problem, 



v(x) = supE x 



e- as f(X s )ds + e- aT >K(X Ti _,X Ti 



(3.8 



and define the set S n and the objective function v n as follows: 

S n = {v e S; v = (7i,T 2 , ...T n+ i;£i,£ 2 , -t,n)\T n+ i = +00}, 

and 



v n (x) = sup 



e~ as f(X s )ds + Y,e~ aT *K(X Ti _,X Ti ) 



(3.9) 



In other words, we are al lowed to make at most n i nterve ntions. For this recursive approach, see, 
for example, Davis (|1992l ) and 0ksendal and Sulem (|2002l ). We use the following simple notation: 



g(x) 4 E x 



/>oo 

/ e- a3 f(X° 8 )ds 
Jo 



(3.10) 



Let Tt denote the space of all Borel functions. Define the two operators M. : Tt — > Tt and C : TL — > 7i 
as follows: 

Mu(x) 4 SU p[K(x, y) - (g(x) - g(y)) + u(y)\, (3.11) 



and 



Cu{x) = supE x [e~ aT Mu(X T _)}, 

TS-S 



(3.12) 



for u £ 7i. From the definition of the two operators, (i\{x) < a 2 (x) for x G M, ai(-), a 2 (-) € TC 
implies Aiai(x) < A4a 2 (x) and Ca\{x) < £a 2 (x) for all x € R. Consider the following recursive 
formula: 



w n+ i(x) = sup E x 



[ T e~ as f(X s )ds + e~ ar (K(X T _,X T ) + w n (X T )) 
Jo 



(3.13) 



which is equivalent to 



w n+1 (x)-g(x) = sup E x [e- aT (K(X T ^,X T ) - g(X T „) + w n (X T ))} (3.14) 

by applying the strong Markov property with (|3.6|) to the integral term. In fact, this derivation is 
explained in detail in subsection 13.21 By defining 

(f> = w - g, 

and adding and subtracting g(X T ) on the right hand side of (|3.14p . it becomes 



<j> n+1 (x) = su P E*[e- ar .M0 n (AV; 

TS-S 
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It should be noted that, for each n, this is an optimal stopping problem over r and can be written, 
by using the operator defined in (|3.12p . 

4> n +i(x) = Ccj) n (x). (3.15) 

Let us start this recursive scheme with wq(x) = g(x) (i.e., no interventions are allowed, equiv- 
alently cj>o(x) = 0) and define recursively <p n (x) = w n (x) — g(x) = C[w n -i{x) — g{x)) = C<p n -i- 
Clearly, 

01 (x) = CM*) = sn V W[e- aT {M{w {X T ) - g{X T ))\ 
res 

= sup E x [e- aT {K{X T „,X T )-g(X T _)+g(X T )}]. 

On the other hand, 

v\(x) — g{x) = sup E x 



roc 

/ e- as f(X s )ds + e~ ar K(X T ^,X T ) 
Jo 



— E x 


[/ 




Jo 



sup E* 



e~ as f{X s )ds + e~ aT [E Xt 
e- as f(X° s )ds-e- aT g(X T _) 



e- as f(X s )ds 



roo 

/ e~ as f(X° s )ds 
Jo 

+ K{X T .,X, 



= sup E x [e- aT {K(X T _,X T )+g(X T )-g{X T -)}]. 

The last equation is due to the fact that only one intervention is allowed. Hence we have wi(x) = 
vi(x). By the definition of the recursive scheme, w n is an increasing sequence (i.e, w\(x) < W2(x) < 
.... for all x E R). In fact, we shall prove that w n = v n for all n in Lemma 13.21 Before that, we 
need the following lemma to relate this recursive scheme with the method described in Section [2J 



Lemma 3.1. The mapping x 



C<t>(x) , jri 

fix) ■ + 



is F -concave. 



Proof. We shall fix some x E (l,r) C [c,d]. Since Mcp(-) is bounded there, for a given e > 0, there 
are admissible e-optimal intervention pairs (o~ l £ ,£, £ ) and (<r£,££) such that 



E l [e~ a <M<f>(X a i )] > C(p{l) - e, and E r [e- aa - M(/)(X a r)] > £0(r) - e. 



Define another stopping time a\ r E 5 with 



Zr A 



T l + a l £ o6 T i, if r'<r r , 

T r + <0^, if T l >T r . 

Putting all together, with the strong Markov property of X, we have 
£4(x) > E x [e~ a ^ r M(f>(X al r)] 

> (£0(0 - e)E x [e-^ l l {rl<T r } \ + (£</>(r) - e)E*[ e — "l {Ti>T , } 

m) F(r)-F(x) C4(x) F{x) - F{1) _ 
~ ip(l) ^ } F{r)-F{l) + <p(r) ^ { } F{r) - F{1) ^ 

Since e is arbitrary, we have an F-concavity. 



□ 
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This lemma guarantees that we can use Proposition 12.11 to 12.31 to identify the value function 
and an optimal stopping rule for each of the recursive optimal stopping problems (|3. 13|) . Let us 
define, for notational convenience, 

K(x, y) 4 K(x, y) - (g(x) - g{y)). (3.16) 

Further, we prove the following properties of the recursive optimization scheme. 

Lemma 3.2. If we define w n by h3.13\) (with wo = g) and v n by 113. 9\) , then 

w n (x) = v n (x) for each n and v(x) = lim w n (x). 

n— >oo 

Moreover, w is the smallest solution majorizing g of the functional equation w — g = £(w — g). 
Proof. The proof is given in Appendix. □ 
Hence if we solve the optimal stopping problem 

<t> n +l(x) =snpE x [e- aT MMXr-)} (3.17) 
res 

recursively for each n, then we obtain cf>(x) = linv^oo (f> n (x) = limn^oo v n (x) — g(x) = v{x) — g(x). 
Summarizing the above argument, we have the following proposition: 

Proposition 3.1. The value function v(x) for \3. 8\) is given by the smallest solution majorizing g 
of the functional equation v — g = C(v — g), and ^£(= i-) is always F -concave. 



Proof. The first statement comes from Lemma 13.21 By the recursive method that we described 
above, we are solving a series of optimal stopping problems for each <j> n . Hence Lemma 13.11 and 
Proposition 12.11 give the second statement. □ 

3.2 Characterization of the Intervention Times and the Value Function: F- 
Concave Reward Case 

Based on the results in the previous subsection, we first consider a special case where the mapping 
x — > ^(x) : — > is F-concave. The argument in the previous subsection is modified to 
incorporate the existence of the ruin state. Instead of (|3.9p and (|3.13p . we define, respectively, 



v n (x) = sup E x 



w n+ i(x) = sup E x 



e- as f(X s )ds + e~ aT »P + e- aT >K(X Ti _,X Ti ] 

Ti<TQ 



r Ar 



with 



w (x) =E a 



e- as f(X s )ds + e- a ^Pl {T0<T} 

+ e- ar {K(X T ^,X T ) + w n (X T )}l {r<To} 

/•oo 

/ e- as f(X°)l {s<To} ds + e- aT °P ±g (x). 
Jo 
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Then by defining the operator £ : 7i -^>TL instead of (|3.12p . 



Cu(x) ± supE x le- ar Mu(X T -)l {T<To} + e - aT °(P - g(0))l {ro<r} }, (3.18) 
res 

we have the same recursion formula as in f)3. 15|) . We can obtain the same results as in Lemma 13. II 
and Lemma [3.21 Proposition 13.11 also holds with one change that the value function is given by the 
smallest solution majorizing go of the functional equation v — g = C(v — g) where C : 7i — * 7i is 
given by (|3.18j) . Now we consider the characterization of the intervention times. 

Proposition 3.2. If the mapping x — > ^(x) : R+ — ► R+ is F -concave and is an absorbing state, 
then the optimal intervention times T* are given, for some b* 6 R +; by 

T* = mf{t>T*_ 1 ;X t $ (0,6*), i = 1,2, ....}. 

Proof. Our proof is constructive, describing the procedure of recursive optimization steps. For any 
n > 1, in view of Lemma 13.11 <f) n (x) is the smallest F-concave majorant of £(f)(x)/(p(x). This 
majorant (that passes (F(0), in the transformed space) always exists. Indeed, since we 

consider the case of & > 0, i.e, x > y for K(x, y) and 

M4> ( x ) = SU P l K ( x , v) - (9(x) - g{y)) + Mv)} = SU P l K ( x , y) - (gfr) - g{y)) + (go(y) - g(y))}, 
we should check whether the concave majorant exists, namely, 

lim(K{x, y) - g{x) + g (y)) <P- g{0) (3.19) 

holds when y J. 0. Note that hm y j ffo(y) = P an d g(x) — ► g(0) as a; — > due to the continuity of 
/. Hence (I3.19P holds in the neighborhood of y = because of (13. 4jl . In the subsequent iterations, 
we consider 

M0i(x) = sup [if(x, y) - (g(x) - g{y)) + <^i(y)]. 

yeR+ 

We should check if the expression inside the supremum operator becomes less than P — g(0) as 
x I and y J. 0. Since lim^o <^i(y) = 0i(O) = P ~ 5(0) by the concavity (hence continuity) of </>i 
and since lim x ^o g{ x ) = ^ m yio g(y), w e have in the neighborhood of y = 0, 

lim K(x, 0) + P - ^(O) < P - g{0) 

holds. Hence the concave majorant always exist also in the subsequent iterations. 

Now the .F-concavity of <p n is obviously maintained for all n. The limit function, 4>(x) = 
limn^oo 4> n (x) exists and is also F-concave. Accordingly, K(x,y)/<p(x) + 4>(y) is F- concave. Hence 
<p and K + 4> meet once and only once. Recall that the value function satisfies 4> = This implies 
that the continuous region is in the form of (0, b*) for some b* G R + , which completes the proof. □ 
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By using the above characterization of intervention times, we next want to characterize the 
value function and reduce the impulse control problem (|3.7p to some optimal stopping problem. 
Moreover, we shall present a method that does not have to go through the iteration scheme. Let 
us first simplify J u : 



J v (x) = E x 



/ e- as f(X s )ds + e- aro P+ V e- aT *K(X Tt _,X Tt 

JO rj, ^_ 



Ti<T 



(3.20) 



This is just a reproduction of (|3.5p . Let us split the right hand side of (|3.20p into pieces and use 
the strong Markov property (together with the shift operator #(•)) to each of them. The first term 
becomes 



e- as f(X s )ds 



E 2 



{Ti<t } 



pi l f'OG 

/ e- as f{X s )ds + e- a V^ / e~ as f(X s )l {s<To} ds 
Jo Jo 

fro 

l{Ti>ro} / f(X s )ds 
Jo 



+ E 2 



The second and third terms become 

E x e -ar 0p = E x h^^e-^^x^-airo-T^p^^ + ^ >To}e ~^0 pj 

= E X [l {Tl<M e- aT m x [e- a ^° e ^P\T Tl ]] +K X [l {Tl>ro}e - aTO P] 
= E* [l {Tl<ro} e- aTl E^i(e-^P)] + E* [l {Tl>To} e~^P] 

and 



E 1 



e- aT >K(X Ti .,X Ti ) 



Ti<ro 



E x 



E x 



h Tl <r } { e- aT ^K(X Tl ^X Tl )+e- aT ^e- a ^' T ^K(X T ^,X Ti )l {Ti<T0} 

i=2 

e- aT ^K(X Tl .,X Tl )+e- aT 'E x [ ^ e~ a ^° e ™ K{X Sl ^ X Si )\T Tl ] 

Ti<ro 

l{T 1<T - }e- aTl \K(X Tl .,X Tl ) +E X ^ ^e- aT ^(X Ti _,X Ti )l {Ti<ro} 



{Ti<r } 



i=l 



where Si = Ti+TjO#(Ti) and the index i runs from 1 for the sum in the second equality. Combining 
the three terms and rearranging, we have 



J v (x) = E 2 



1{Tl<To} {C e ' aSf{Xs)ds + ^K&Tl-iXtJ + e- QTl ^(X Tl )| 



+ E 2 



hTi>r }\ I e- as f(X s )ds + e- aT0 P 



(3.21) 
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For any F stopping time r, the strong Markov property with our assumption f|3.6|) gives us 



W 



e- as f(X®)ds 



9 {x)-W [e- ar g(X° T )] 



where g(-) is defined as in (I3,10p , We apply this result to (I3.2ip by reading r = T\ and r = To to 
derive 

r(x) = E x [l {Tl<To} e- aT i{K(X Tl „,X Tl ) - g{X Q Tl ) + J v {X Tl )}] 

+ E* [l {ri>To} e-^{^-5(^r )}] +9(x). (3.22) 
Noting that g(Xj, ) = g(Xx 1 -, adding and subtracting g{Xx 1 ) and further defining 

u(x) = J v {x) - g(x), 

(|3.22p finally becomes 

u(x)=E* [l {Tl<To} e- aT i{K(X Tl „,X Tl ) + u(X Tl )-g(X Tl ^+g(X Tl )}] 

+ W [l {Tl >r 0} e~ aT °{P-g(X TQ )}] , (3.23) 

and we consider the maximization of this u(-) function and add back g(x) since supit(x) = 
sup{J^(x) — g(x)} = sup J v (x) — g(x). Note that this simplification leading to (|3.23|) does not 
depend on the i^-concavity assumption. 



Since we have confirmed that optimal intervention times are exit times of the process from 
an interval, let us use a simpler notation that X^~ = b% and X^ = di for all i. We can denote 
Ti- = r b = mf{t >0;X t > b { }. By observing (|5^5|> . 



u(b) = u(X Tl _) = K(X Tl „,X Tl ) + g(X Tl ) - g(X Tl _) + u(X Tl ) 

= K(b, a) + g(a) - g(b) + u(a) = K(b, a) + u(a) 
u(0) = u(X T0 ) =P- g(X T0 ) = P- 5 (0) 



we have 



u(x) 



u (x), x G [0,6) 

K(x, a) + uo(a), x £ [b, oo). 



where 



(3.24) 



(3.25) 



u (x) 4 E-[l {Tb<To} e-^u(b)]+E-[l {Tb>To} e~^u(0)l 



The second equation of (|3.25p is obtained from (|3,23p by noticing that, on a; G [6, oo), F x (Ti < 
to) = 1. Indeed, in this case, we immediately jump to a, so that Xt x - = x and Xt 1 = a. Since 
a £ (0, 6), u(a) = uo(a). Now let us note that we have the following representations in (|3.23p 
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where 77 = inf{i > 0; Xt = 1} and r r = inf{t > 0; Xt = r} and </?(•) and tp(-) denned in the previous 
section. Finally, with being defined as in (|2.2p . we have a characterization of u(x), 

u{x) u(b)(F(x) - F(0)) n(0)(F(q)-F(x)) 

y?(x) ^(6)(F(6) - F(0)) + m(F(b) - F(0)) ' ^ 
Define W = o F^ 1 , this becomes, for any a and 6, 

^(F(x)) = ^(F(6)) : *® + W(-F(0)) g| I , x€[0,6]. (3.27) 

This represents a linear function that passes a fixed point, ^4 = (F(0),W(F(0)). 

To discuss how to find the optimal pair (a*, b*), we write n(x) as u a ^{x) to emphasize the 
dependence on a, 6, then on x E [0, 6], 

sup u a>b (x) = sup sup{E x [l {r6<To}e - aT6 (E:(&,a) + u a , fe (a))] +R x [l{ n>7rs }e~ a7 *u a>b (0)]}. 

aeR+6eR+ aeR+fteR+ 

(3.28) 

This can be considered as a two-stage optimization problem. First, let a be fixed. For each a, the 
inner maximization of (|3.28[) becomes 

V a [x) 4 S u V {W[l {n<To} e-^{K{b,a) + V a {a))] +E x [l {Tb>To} e~- a ^(P - g(0))]} (3.29) 
r b es 

and, among a's, choose an optimal a in the sense, v(x) = sup a V a (x) for any x. It should be pointed 
out that V a (x) may take negative values if P — g(0) does. Now, we discuss a solution method of 
the first stage optimization (|3.29p . For this purpose, we need a lemma: 



Lemma 3.3. If we define 



G(x, 7 ) = supE x [e- aT {h(X^) +7)], x E M, 7 E R 
res 

for some Borel function h : R — ► M and luit/i condition \3. 6\) , then, for 71 > 72 > 0, 

G(x, 7l )-G(x, 72 ) < 7 i-72, (3.30) 

/or any x. 

Proof. The left hand side of (|3.30p is well-defined due to (|3.6p . It is clear that G(x,7) is convex in 

Hi 70 

G(x,7i) - G(x,7 2 ) 



7 for any x. Then Z?+G(x,7o) = lim 7 , 70 G ^' 7 °^_^ x ' 7 ^ exists at every 70 E R, and 



7i - 72 

Consider the bound of G(x, 7) for x fixed: 



<D+G(x,7i). (3.31) 



G(x, 7 ) < supE x e- ar |/i(X r )| + | 7 | supE x e- QT . 
t<=s res 

The first term on the right hand side is constant in 7 and the second term is linear in 7 and 

the E x [e~ Qr ] < 1 for any r E S. Due to the convexity of G(x, / y) in 7, for the above inequality to 

hold, D+G{x, 7) < 1 for all 7 E R. On account of (jOT]) . we have (13301) . □ 
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Coming back to (|3.29p . we need some care because the value function V a {x) contains its value 
at a, V a (a) in the definitive equation. Let us consider a family of optimal stopping problem 
parameterized by 7 € R. 

^(^^suplE^l^^e-^^^^^+^+E^l^je-^CP-yCO))]} 
res 

= supE^e-^r 7 ^, a)] (3.32) 
res 



where 

r 7 (x, a) 



P-g(0), x = 0, 
K(x, a) + 7, x > 0. 



Obviously, this parameterized problem can be solved by using Proposition 12.11 to 12.31 Now we link 
this parameterized optimal stopping problem to (|3.29j) . 

Lemma 3.4. For a > given, if there exists a solution to A3.32\) . then there always exists unique 
7 such that 7 = Va(a) holds, provided that \3.4\) holds. 



Proof. Without loss of generality, we need only to consider the case where 

supK(x,a)>0 (3.33) 

for some a > 0. Indeed, suppose that there is no such a and let us consider a sequence of optimal 
stopping scheme. In each iteration, the value function for the optimal stopping problem takes 
negative values, so that 4> n (-) < for all n. Then in the next iteration, K(x,y) function will 
be shifted downwards, leading to </> n +i(-) < 0. Hence the "no interventions" strategy is trivially 
optimal. 

In (|3,32p . since 7 is some constant parameter, we benefit from Proposition 12.11 and claim that 
Va(x) is characterized as the smallest F-concave majorant of r 7 (-,a) that passes ^F(0), 
In terms of the notation of Proposition 12.31 if we define W2{-) such that 



K 7 (x) = ^(x)W^F(x)), 

then Wa(-) passes through the fixed point A = (F(0), W2 (F(0)) and is the smallest concave 

majorant ot rl >{-,a) - ^ F -i^ - v (f~H-)) + <p(F- l (-))- 

Now fix a. Our approach here is by starting with 7 = 0, we move 7 and evaluate Va(a) and try 
to find 7 such that 7 = Va(a). Due to (|3.33p . we have W^{F(a)) > 0. By the monotonicity of F, 
it is equivalent to saying that V®(a) > = 7. As 7 increases, W2{F{a)) increases monotonically 
by the right hand side of (|3.32[) . Lemma [3731 implies that for 71 > 72 > 0, 

V2 fl (x)-V2 f2 (x)<7i-72 (3.34) 

for any x £ R+. Note that Wj(F(a)) > H"<(F(a),a). However, since V has less than the linear 
growth in 7 as demonstrated by (|3.34[) . there is a certain 7' large enough such that W2{F(a)) = 
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H' y (F(a),a) for 7 > 7 . This implies 

y{a)W2{F(a)) = y{a)H\F{a),a) 
&Vj{a) = K(a,a)+i < 7' 

where the inequality is due to the assumption (13,41) , For this 7 , we have Va (a) < 7 • 

The monotonicity and continuity of W" / (F(a)) (due to the convexity of Va {■)) with respect 
to 7, together with (|3.34p . implies that, for any a, there exists one and only one 7 such that 
V2(a) =7. □ 

3.3 Methodology to find v(x) and (a*,b*) 

Using (|3.26j) . namely the characterization of u a ^, we describe an optimization procedure based on 
Proposition [2JZ] and EES 

1. Fix a. Consider the function 

A K(F~ 1 (-),a) 

Define W a (-) such that V a (x) = ip(x)W a {F(x)) and by the characterization (|3.20p . it is a 
straight line with a slope, say (3(a) and passes through (F(0), W a (F(0)) = ^F(O), P ~^ ^j ■ 
We can write the linear majorant, in general, 

W a (y) = (3(a)y + d. (3.36) 

2. First stage optimization: For each slope /3(a), we can calculate the value of W a (F(a)), but we 
have to find the W a (-) function such that, at some point F(b(a)), we have 

Wa (F(b)) = R(F(b),a) + W a (F(a))^. 

where we write b(a) = b for notational simplicity. This requirement is equivalent to finding 7 
in (|3.32[) in Lemma 13.41 such that 

= W2{F(a)). 

By Proposition 13.21 C a = (0, 6(a)) is the continuation region. If R is a differentiate function 
with respect to the first argument, we can find the optimal point 6(a) analytically. In effect, 
it is to find a point 6(a) such that the linear majorant and the shifted function R(F(x),a) + 
W a (F(a))^pj have a tangency point. This is equivalent to calculating the maximum slope 
that majorizes R(F(x),a) after it is shifted. Explicitly, we solve 



for 6(a) where /3(a) is 



y{bf 



/3(a) = ^T^'^r^- (3-38) 
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For the absorbing boundary case, these equations can be easily modified. Let us denote 
D = W a (F(0)) = (P - g(0))/(p(p). Then (IBTBTj) and (13351) become 

and 

Pl J (F(b) - F(0)Mb) - (F(a) - F(0)Mo) ' 1 ' j 

respectively. 

3. Second stage optimization: Now, let a vary and choose, among /3(a), find 0*, if exists, and 
also the corresponding b(a*) and a*. Due to the characterization of the value function, these 
a* and 6* = b(a*) must be the solution to (|3.7|) . Suppose that a) is a decreasing function 
of a. As a becomes closer to 0, the quantity K(x, a) becomes larger, while W a (F(a)) smaller. 
Hence we can expect the existence of a* that maximizes the slope (5. 

Remark 3.1. With respect to the third point of the proposed method above, we should check if 
there exists a concave majorant as a { 0. Namely, we consider whether 

lim(K(x, a) - (g(x) - g(a)) + u(a)) < P - g(0) 

holds in the neighborhood of a = 0. Since lim^o g(x) = nm a j,o g(o) and lim a ^o u(a) = u(0) = 
P — a(0) by the continuity of u, the last inequality holds due to (|3.4|) . 

3.4 Characterization of the Intervention Times and the Value Function: Gen- 
eral Case 

Let us move on to a general case where the mapping x — > ^(x) : R+ — > R+ is not necessarily 
P-concave. First, we extend Proposition 13.21 to characterize optimal intervention times. 



Proposition 3.3. The value function v(x) for \3. 7| ) is given by the smallest solution majorizing g 
of v — g = C(v — g) and optimal intervention times T* are given by exit times from an interval if 
and only if, for all y E R + , 

x — ► K(x,y) is continuous and q = limsupP~ ( — o F _1 J (x) is finite.. (3-41) 



where D f(xo) = limsup^ f( x o) 



x^xo 



Proof. For any given a £ R+, if we can find the smallest linear majorant of ^^rp=Trjy^ f° r an 
arbitrary 7 G R + , we can find 7 = (p(a)W a (F(a)) by Lemma [331 Due to the constancy of 7, it 
suffices to show that condition (|3.4ip is necessary and sufficient for the existence of concave majorant 



of o F 1 on F(Z). The sufficiency is immediate. For the necessity, we assume that q = +00. We 

1- (Krs J7-A („k\ 



can take a sequence of points {x } C R such that x — > 00 and D I — o F 1 J (x ) — > 00 as A; — ► 00 



14 



If necessary, by taking a subsequence, we can make this sequence {x k } monotone. Consider the 
smallest concave majorant of ^ o F^ 1 on [F(0), F(x k )]. Call it v k (x). It is clear that v k (x) is 
monotone increasing in k for all x G [F(0), F(x k )]. As k — > oo, x fc — » oo and v(x) > v k (x). We 
thus have v(x) = ]im.f.—> 00 v k (x) = oo for all x G M+. There is no optimal intervention policy. □ 

Suppose that the P-concavity of the reward function is violated, so that the intervention point 
may be multiple. Let us consider a strategy that we have two intervention points, b\ and 62 being 
arbitrarily chosen such that < b\ < b 2 - We want to characterize function J v (x) as in (|3.5p again. 
Recall that there are no controls in a way that the process is pulled up to avoid ruin. In other 
words, P^fro < 00] = 1. Assume, for the moment, that we always apply control at these boundaries 
61 and 62 and then, once applied, the process moves to a\ < b\ and 02 < 62, respectively. 

If we start with a point x G [0, 61], the problem is equivalent to the case we considered already, 
since the process cannot go beyond the level b\. Hence following (|3,25p . we have for x G [0, 61] 



J\(x) = E x 



TO 



e- as f(X s )ds + e- aT0 P+ 2^ e 



Ti<ro 



-aTi 



and 



Ul (x) =E-[l {r6i<To} e-^i Wl (6 1 )] +E*[l {r6i>To} e-^m(0)], 



xe [OA] 



by defining u\{x) = 
available: 



Ji(x) — g(x). If we start with a point x G [61,62]) there are two strategies 



(A) Let X t move along. (It either hits b\ or 62 first.) 

(B) Apply the control immediately (t = 0) by moving the process from x to a\ (the post-control 
point that corresponds to 61) and let the process start at a\. (Recall that we do not let X 
enter into (61,00) after moving to a±.) 

Consider strategy (A) first. Let us define 



r i>1 



e- as f(X s )ds + 



E 

Ti<r bl 



e- aT >K(X T ^,X Ti 



x G [61,62]. 



Using the strong Markov property, we can reduce to a simpler form. For any (01, 61) and (02, 62), 
we have 

r 2 {x) = ri^^je-Hif^,^)- 9(X Tbi ) + Jax ni )+g(X Ui )- g(X ni _) 

+ W[l tTbi>Tb2} e- a ^K(X n2 _,X T J-g(X Tb J + J^X^ 

We shall use u\(x) = J\{x) — g(x) in the first term. Now let us define 1*2(2;) — J^i 30 ) ~ g{ x )- Then 
the last equation becomes 

H K(X Tbi _, X Tbi ) + Ul (X Tbi ) + g(X Tbi ) - g(X Tbi _) 



u 2 {x 



+ mhr bl >r b2 }e- a ^K(X Tb2 _, X Tb2 ) + u 2 {X Tb2 ) +g{X Tb2 ) - g{X Tb2 _)] 
E x [l {ni<Tb2} e- aT h(K(b 1 ,a 1 ) + u 1 (a 1 ))]+E*[l {T ^^ (3.42) 
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on x e [h, b 2 \. By identifying K(b 2 , a 2 ) + u 2 (a 2 ) = ^2(62) and U2(6i) = K(bi, a{) -fui(ai) = 
that shows U\(x) and U2(x) are connected at x = &i. Thus, 

¥>(6l) F(6 2 ) - -F(Ol) ^(02) F(02) - F(6i) 

To summarize this result, if we define W f (.) = ^of-^.) for i = 1,2 on F(T), this is again a linear 
function for each i. Hence by defining 

\w 2 {F(x)) = W 2 (F(b 1 )) ^ ) :^ ) +W 2 (F(b 2 )) ^Z^y xefoM, 

we have a piecewise linear function on F(T). Moreover, since we can treat 61 as an absorbing 
boundary, we have a\ < b\ < a 2 < b 2 . 

Next consider strategy (B), whose value function is 

A (Wi(F(x)), 0<x<bi 
WW)= _ / - - 1 (3.44) 

(W^Fix)) 4 5^Wi(F(ai)) + R(F(x), ai ), h < x. 

Lemma 3.5. (A) is better than (B) only if 

4 W{F{b{)) - W(jm) W{F{b 2 )) - WjFiPx)) 4 
Pl F(6i)-F(0) F(b 2 )-F(bx) ' 2 - 

Proof. Since the value function of strategy (B) is (|3.44p . choosing (A) over (B) is equivalent to 

Wi{F{x)) < W 2 {F{x)) on x > b x . 

If W\{F{x)) majorizes W\{F{x)) on x 6 [0, 00), then this problem reduces to F-concavity case 
discussed in the previous subsection. Hence we consider the case where there exists some x £ [61, 00) 
such that 

Wi(F(aO) <W[{F{x)). 

Now suppose that we have j3\ < j3 2 . Then it is clear that we cannot have W 2 {F{x)) > W\(F(x)) 
on x £ [b\, 00). □ 

There are two cases to consider: 

(1) If W 2 (F(x)) majorizes W\{F{x)) on x G [61,00), then we adopt the point b 2 as an in- 
tervention point. In this case, (3 2 > f3\ holds. However, this implies that if we connect 
A = (F(0),Wi(F(0)) and C = (F(b 2 ),W 2 (F(b 2 )), then this line segment AC is above the 
line segment connecting, piece by piece, points A, B = (F(b\), W\(F(b\)) and C. We can 
show that there exists a point b' > b 2 such that its corresponding linear majorant W'(F(x)) 
satisfies W'{F{x)) > Wi(F(x)) on x e [0, 61] and W'(F(x)) > W 2 (F(x)) on [61,62]- The 
proof of the existence of a post-intervention point a' corresponding to this point b' follows in 
a similar manner to Lemma 13.41 
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(2) If W2(F(x)) does not majorize W\(F(x)), we can find another point 6, instead of b\, such that 
the linear (not piecewise linear) function W(F(x)) corresponds to b majorizes R(F(x),a) + 
W(F(a))$$ on x £ R + by Proposition E3J 

In either case, the value function in the transformed space should be a linear function that attains 
the largest slope among all the possible linear majorant. This argument holds true for any b\ and 
62 with b\ < 62. We can continue this argument inductively to the case of n intervention points, 
(61, ...b n ). We here summarize our argument up to this point as a main proposition: 

Proposition 3.4. Suppose that $j3.41\ ) holds and the optimal continuation region is connected. The 
value function corresponding to \3. 5\) of the impulse problem described in \3. 3]) ~ (3.1) is written as 

fv (x)±tp(x)W*(F(x))+g(x), 0<x<b* 
v[x) = < (3.45) 
(v (a*)+K(x,a*), b* < x. 

where W*(-) is the line segment connects (F(0),W*(F(0))) and (F(b*),W* (F(b*))) and satisfy the 
following: 

1. W*(F(-)) is the smallest linear majorant of W*(F(a*))^y + R(F(-),a*) and meets with 
W*(F(a*)) + R(F(-),a*)?$7 at point F{b*) and passes (F(0),^W). If R is differentiable, 
(a*,b*) satisfy (Egg J. 

2. The slope ofW*(-), denoted as (3* , is the largest slope among (3(a) 's of all the possible linear 
majorants W a (-). 

Moreover, if the mapping x — > — (x) : R+ — ► is F -concave, then the optimal continuation region 
(0,6*) is uniquely determined. 

Note that, at x = 0, 

v(0) = v(0)W*(F(0))+g(0) = y(0) P "; (0) +ff(0) = P 

as expected. 

Remark 3.2. If the F-concavity of K is violated, there are two possible cases (and combination 
of them) of multiple continuation regions. 

1. For some a* with i = 1, 2, we have the common (3* . This is the case which we shall show 
in the next example. In this case, the continuation region is C = {(0,6*), (&*,&*,)> (^2>^3)---} 
where 6* corresponds to a* for each i, and the intervention region is T = {{6J}, {6?;}, {63}...}. 
Each time the process hits one of the points {6*}, the control pulls the process back to the 
corresponding a*. 

2. Another case is that, for the unique optimal a* , there exists non-unique 6* and b%- In this case, 
the continuation region is C = {(0, 6*), (6*, 6?,)}, and the stopping region is T = {{6^}, [6?,, cxd)}. 
If the process hits 6* or b%, then the control pulls the process back to a* in either situation. 
It makes sense to continue in the region (6*,&*>) because there is a positive probability that 
one can extract K(b2,a*)(> K(b*,a*)) within a finite time. 
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3.5 No absorbing boundary case 



Next, we extend our argument to a problem without the absorbing boundary. Hence the process 
can move along in the state space in an infinite amount of time. The problem becomes 



J v (x) = E a 



/•OO 

/ e— f(X s )ds + J2e~ aTi K(X Ti _,X Ti 



(3.46) 



We can characterize intervention times as exit times from certain boundary and simplify the per- 
formance measure (|3,46p 



J v (x) = E x 



/•oo 

/ e- M f(X s )ds + Y,e- aTi K(X Ti _,X Ti , 
Jo ~1 



= E x [e- aT ^{K(X Tl .,X Tl ) - g(X Tl .) + F (X Tl )}} + g(x). 

The second equation is easily obtained in the same way as in the previous section by noting 
F x (Ti < oo) = 1. The last term does not depend on controls, so we define u{x) = J u (x) — g(x): 

u(x) =E x [e- aT '{K(X Tl .,X Tl ) - g(X Tl _) + g(X Tl ) + u(X Tl )}}. 

Again, we consider the -F-concave case with the notation Tj— = 7& for all % and we have 

u(x) =E x [e- aTb {K(b,a) - g{b) + g{a) + u{a))} = E x [e~ aTb {K(b, a) + u(a))]. 

By defining W = (u/ip) o F , we have 

W(F(x)) = W(F(c)) ^~ ffij + W{F{b)) F ^~^ C ] , x e (c, b}. 



F(b) - F(c) 



F{b)-F(c) 



We should note that F(c) = F(c+) = if)(c+)/(p(c+) = and 

W(F(c)) =l c ± limsup K{X ] Q } + 

xlc <P{X) 

for any a G (c, d]. For more detailed mathematical meaning of this value l c , we refer the reader to 
Dayanik and Karatzas (120031 ) . We can effectively consider (F(c),l c ) as the absorbing boundary. 



4 Examples 

In this section, we work out some examples from financial engineering problems. For this purpose, 
we recall some useful observations. If h(-) is twice-differentiable at i 6 I and y = F(x), then 
H' (y) = m{x) and H" (y) = m (x)/F (x) with 



m(x) = -^—(-)(x), and H" (y)(A — a)h(x) > 0, y = F(x) (4.1) 
F (x) \<p 1 



with strict inequality if H (y) ^ 0. These identities are of practical use in identifying the concavities 
of H (•) when it is hard to calculate its derivatives explicitly. 
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Example 4.1. 0ksendal (|1999l ) considers the following problem: 



J»(x) = E x 



/'OO _^ 

/ e- Qs l> + Ve- aTi (c + 
Jo 



(4.2) 



where X® = Bt is a standard Brownian motion and c > and A > are constants. The Brownian 
motion represents the exchange rate of some currency and each impulse represents an interventions 
taken by the central bank in order to keep the exchange rate in a given target zone. Here we are only 
allowed to give the system impulses £ with values in (0, +oo). By reducing a level from b to a (i.e., 
b > a) through interventions, one can save continuously incurred cost (which is high if the process is 
at a high level). The problem is to minimize the expected total discounted cost v Q {x) = inf^ J^{x). 
We want to solve its sup version and change the sign afterwards (i.e. v (x) = —v(x)): 



v(x) = supE 3 " 



f'CG x 

/ e~ as (-X^ds-Y, 
Jo 



e- aT *(c + A& 



The continuous cost rate f(x) = —x 2 and the intervention cost is K(x,y) = —c — X(x — y) in our 
terminology. By solving the equation (A — a)v(x) = \v" {x) — av(x) = 0, we find ip{x) = 



and cp(x) = e' 



Hence F(x) = e 2xV2a and F~ 1 (x) 



2V2a 



Following our characterization of 



the value function, we obtain 

r(x) = E*[ e -^ {K(X Tl ^,X Tl ) - g(X^_) + r(X Tl )}]+g(x) 
where g{x) can be calculated by Fubini's theorem: 



g{x) 



-E 2 



(x + By<is 



+ 



1 



a 



a- 



By defining u(x) = J v {x) — g(x), we have u{x) = E, x [e aTb {K(b,a) — g{b) + g(a) + u(a)}]. Note 
that when b > a, g(a) — g(b) > is the source of cost savings. 



Let us fix a > and consider h(x) 



-c — X(x — a) + 



and H(y) ± {h/^){F-\y)\y > 



(x) will lead us to conclude that H(F(x)) is 



0. By the first equation in (|4.ip . the sign of 
increasing from a certain point, say x = p on (p, oo), so is H(F(x)). Also, by direct calculation, 
H'(+oo) = 0, from which we can assert that the value function is finite by Proposition 13.31 If we 
set p(x) = —x 2 + a 2 + \a(x — a) + ac + 1/q, then (A — a)h(x) = p(x) for every x > 0. This 
quadratic function p(x) possibly has one or two positive roots. Let k be the largest one. Since 
lim 3 ,^ 00 p(x) = — oo, by the second inequality in (|4.ip . H(-) is concave on (F(k), +oo). Hence 
H(y,a) is increasing and concave on y £ (F(k), oo). Since the cost function in the transformed 
space is increasing and concave from a certain point on, there is a linear majorant that touches the 
cost function once and only once. We can conclude that for any a > and the parameter set, we 
have a connected continuation region in the form of (0,b*). 

For this fixed a, let us define W a {-) such that V a (x) = ip(x)W a (F(x)) and r(x, a) = — c if x < a 

2 2 

and — c — A(x — a) + ^- — 2_ if x > a. Then we have for any a > 0, 

r(x, a) 



L 



lim sup ■ 

x[— oo 



tp(x) 



0. 



(4.3) 
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Recall that the left boundary — oo is natural for a Brownian motion. Hence W a (y) that passes the 
origin of the transformed space is the straight-line majorant of R(-, a) + W a (F(a))/if(F~ 1 (•)) where 
R(-, a) is defined in (|3.35|h 



R(y,a) 



-cy/y, 

H(y,a) = y/y (-c - j^ l ogy + Xa + 



(logy) 2 

8a 2 



< y < F(a) 
V>F(a). 



We can represent W a as W(y) = f3y. Since R(x,a) is differentiable with respect to x on x > a, 
we can use (|3.37p to find b(a) and corresponding (3(a). Then varying a, one can find the optimal 
(a*, b* , (3*). Going back to the original space, on x G (— oo, b*\ 

v(x) 4 supu(x) = <p(x)W*(F(x)) = <p(x){(3*)F(x) = (3*e xV ^. 

To get v(x) = supj, J w (x), we add back g(x), 



v{x) = v{x) + g(x) = (3*e xV ^ - 
Finally, flip the sign and obtain the optimal cost function 

v (x) = 



x l 1 

— + — 

a or 



'""'•"J - ^ + ^ ) " ^e^, < x < b*, 



v (ci*) + c + X(x — a*). 



b* < x. 



which coincides with the solution given by 0ksendal (|1999l ). Figure 1 displays the solution with 
parameters (c, A, a) = (150,50,0.2). 



Example 4.2. This example is a dividend payout problem where the underlying process f ollow s 
an Ornstein-Uhlenbeck process. This problem was originally studied by Cadenillas et al.(j.2003) 
in an ingenious way, but the existence of the finite value function and the connectedness of the 
continuation region were left open. Suppose that X° has the dynamics 

dX? = 6(m - X t )dt + adW t , t > 0, 

where 5 > 0, a > and m G R. Only positive impulse is allowed in this problem. We consider the 
impulse control problem, 



v[x 



sxipE x 



Ti<ro 



with some positive constant K, k and the risk-aversion parameter 7 G (0, 1]. Since £ G R+ , we have 

K(x, y) = k(x — y) 1 — K, x > y > 0. 

The functions ip(-) and </?(•) are positive, increasing and decreasing solutions of the differential 
equation (A — a)v(x) = (l/2)a 2 v" (x) + 5(m — x)v (x) — av(x) = 0. We denote, by ?/>(•) and <p(-), 
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R(F(.),W(F(.)) 




(c) (d) 

Figure 1: (a) The plot of (3(a) against a, the former being maximized at a* = 5.077 with 0* — 0.0492. (b) 
The functions R(F(-),a*) shifted by the amount W a *(F(a*))^A (lower curve) and the majorant W a *(F(-)) 
(upper curve) corresponding to a*, giving us b* = 12.261. (c) The cost function v (x). (d) The derivative of 
v (x), showing that the smooth-fit principle holds at b*. 



the functions of the fundamental solutions for the auxiliary process Zt — (Xt — m)/a,t > 0, which 
satisfies dZ t = —5Z t dt + dWt- For every x£l, 

$(x) = e 5x2/2 V_ a/s (-xV26) and (p(x) = e 5x2/2 V_ a/s (xV26), 

and ip{x) = ip((x — m)/a) and tp(x) = <p((x — m) / 'a) , where T> u (-) is the parabolic cylinder function; 
(see Borodin and Salminen (2002, Appendices 1.24 and 2.9) and Carmona and Dayanik (2003, 
Section 6.3)). By using the relation 

V v (z) = 2-»/ 2 e- z2 / 4 H u (z/V2), zeR (4.4) 

in terms of the Hermite function 7i v of degree v and its integral representation 

H„(z) = —, r / e-* 8 - 2 *^-"- 1 ^, Re(i/) < 0, (4.5) 

i Jo 

(see for example, Lebedev(1972, pp284, 290)). Let us consider the function 

h(x) = kx' y -K, x > 0,7 € (0,1]. 

Since the function h(-) is increasing, the function H(y) = (h/ip) o F^ 1 (y),y £ (0, oo) is also 
increasing. Let us define the function 

p(x) = -cf 2 k^(^ — l)x 7 ~ 2 + m5kjx' Y ~ 1 — k(5j + a)x 7 + aK 
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which satisfies (A — a)h(x) = p(x). By using (|4,ip . H (y) and p{F~ l {y)) have the same sign at 
every y where h is twice-differentiable. Hence we study the (positive) roots of p{x) = 0. We have 
to divide two cases: (1) 7 = 1 and (2) 7 < 1. In either case, it can be shown that ij'(+oo) = by 
using (|4.4j) and (14. 5ft and the identity Ti! v {z) = 2u'H v -i(z), z G M. Therefore, by Proposition 13.31 
the finiteness of the value function is proved. 

(1) 7 = 1: h(-) reduces to a linear function and the p(x) = always has a one positive root, say 
p > 0. H(-) function is convex on [0, F(p)) and concave on (F(p), +00). Hence we have a 
connected continuation region (0,6*). 

(2) 7 < 1: We observe that lim^o p( x ) = ~~ 00 > nm zt+oo p(x) = —00, lim^op'(^) = +00, and 
linx^+oo p{x) = 0— . A direct analysis of p (x) shows that there is only one stationary point 
in (0, 00) and the number of the roots of p(x) = is either 0, 1 or 2. Hence in the first 
two cases, H(-) is concave on [0, 00) and the continuation region is connected. In the last 
case where there are two roots, say < p% < p%- The H(-) function is then concave on 
[0, F(pi)) \J(F(p2), +00) and is convex on (F(pi), F(p2)). Since H(-) increases and concave 
on y E (F(p2), 00), we can conclude that the continuation region is connected in this case as 
well. 

Let us move on to finding an optimal continuation region. By fixing a > 0, let us define r(x, a) = 
if x = 0, —K if < x < a and k(x — a) 7 — K if x > a. When we solve (|3.37p for this a, it is not 
easy (at least analytically) to solve F~ 1 (y) explicitly. We can bypass this difficulty by using the 
first identity of (14. lj) so that (I3.39h with D = becomes 

( KM V KM f„ l(h s <f/(b)ip(a) \ 

KlW) ~ <p(b)(F(b)-F(0))-<p(a)(F(a)-F(0)) [ F (&) + -^T^ " ^ 

As in the previous examples, W a (-) is a straight line passing (F(0),0) in the form of W a (y) = 
(3{y — F(0)). The value function v{x) in x £ (0, b*) is 

v(x) = (p(x)W(F(x)) = f3(F(x) - F(0))cp(x) 

= pMx) - F(0Mx)) = pe* ^ |p„ q/5 (- (^) >fis\ - F(0)V_ a/s ((^) y/2S\ } • 
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Therefore, the solution to the problem is 

v(x) = 



v(x), < x < &*, 

v(a*) + k(x — a*) 7 — K, b* < x. 



This solves the problem. See Figure 2-(b) for the value function in case of parameters 5 = 0.1, 
m = 0.9, a = 0.35, a = 0.105 for the diffusions. As for the reward/cost function parameters, 
k = 0.7, K = 0.1 and 7 = 0.75. The solution is (a*,b*,0) = (0.2192,0.6220,0.5749). 

Example 4.3. We show a simple example where we have multiple continuation regions, the first 
case of Remark 13.21 Let the uncontrolled process is a standard Brownian motion Bt and let a = 0, 
/ = and 

K(x, y) = — c(sin x — sin y) — 5 
with c G R+ and 5 G M+ being some constant parameters. We want to solve 



v(x) = supE x 



E (6 - 5 ) 

7i<To 



In this case = x and let us define 



R(x, a) = r(x, a) 



0, x = 0, 

-c(sinx — sin a) — 5, x > 0. 



By solving (|3.37j) with some parameter (c, 5) = (10,0.35), we find that a* k = 2.75 + Akir and 
b* k = 3.52 + Akir with k = 0, 1, 2.... For all these pairs, /3* has a common value of 9.30. Hence all 
these pairs are optimal. This implies that if the initial state x £ (b k , b k+1 ), then we let the process 
move until it reaches b* k or b k+1 . If it reaches b* k first, then an intervention is made to a* k . Now we 
are in the interval (b* k _ 1 , b* k ). We continue until the process is absorbed at x = 0. 



5 Conclusions 



Before we conclude this article, we shall mention an immediate extension to two boundary impulse 
control problems: 



r{x) =w 



/ e- as f(X s )ds + £ e-^d(X Ti _, X Ti ) + £ e - aS *C 2 (X Sj -, X Sj 
Jo i=i 7=1 



and 



?j(x) = sup J u (x) = J u (x) 



(5.1) 



(5.2) 



for all x G R, where 



= (Ti,r 2 ,....;Ci,C2, ■■■■;Si,S 2 , ....;rji,ri2, 
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with Ci > corresponds to interventions at the upper boundary at intervention time Tj and r/j < 
at the lower boundary at intervention time Sj. 

Examples of this type include the storage model analy zed by Harrison et al. (|1983l ) and foreign 
exchange rate model studied by Jeanblanc-Picque (|1993l ). The former problem, for example, is 
that a controller continuously monitors the inventory so that the inventory level will not fall below 
the zero level. He is allowed to make interventions by increasing and decreasing the inventory by 
paying costs associated with interventions. In this case, the process remains within some band(s). 
In other words, the optimal intervention time s are characterized as exit times from an interval in 
the form of (p* , b*) for < p* < b* . See Korn (|1999l ) for survey. 

Under suitable assumptions, we can develop a similar argument to the previous chapters. Among 
others, the intervention times can be characterized as exit times from an interval (p*,b*). We can 
also simplify the performance measure, 



r( x ) = E-[i {Tl<5l} e" a ^{Ci(x Tl _,x Tl ) - g (x Tl .) + r(x Tl )}\ 

+ E x [l {Tl>Sl} e^{C 2 (X s ^,X Sl ) -g(X Sl -) + r(X Sl )}]+g(x) 

where g(x) = K x e~ as f(X®)ds as usual. Again, the last term does not depend on controls, we 
define u(x) as u(x) = J v {x) — g(x), 

u(x) = E*[l {Tb<Tp} e-^u(b)] +E*[l {7l>Tp} e- on > u(p)], x £ \p,b] (5.3) 
where T\ = r\, and Si = r p and it follows that 

u(x) = u(b)(F(x)-F(p)) u(p)(F(b) - F(x)) 

<p(x) V (b)(F(b)-F(p)) + <p(p){F(b) -F{p)Y [P > h [ ■ 1 

Hence if we define W = ^ o F^ 1 , we have linear characterization again in the transformed space; 

W(F(x)) = W(F(b)) ~_ EM + W(F( P) ) F ^ I , X e\p,b]. (5.5) 
and the solution to the problem is described as 



u(x) 



C 2 (x,q) +u (q), x<p 
u (x) ± W[l {Tb<Tp} e-^u(b)} +E x [l {n>Tp} e-^u{p)}, p<x<b 
C\(x, a) + uq(o), b<x 



where Cj(x, y) = Ci(x, y) — g(x) + g(y) for i = 1 and 2. 

We have studied impulse control problems. The intervention times are characterized as exit 
times of the process from a finite union of disjoint intervals on the real line. A sufficient condition 
is given for the connectedness of the continuation region. The value function is shown to be linear 
in certain transformed space and a direct calculation method is described for it. This method can 
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handle impulse control problems with non-smooth reward and cost functions. The finiteness of 
the value function is shown to be equivalent to the existence of a concave majorant of a suitable 
transformation. The latter is easier to check by using geometric arguments. 

The new characterization of the value function and optimal strategies can be extended to 
other optimization problems, such as optimal switching, singular stochastic control and combined 
problems of optimal stopping and impulse control. If an optimal strategy exists in the class of 
exit times, then the problem can be reduced to a sequence of optimal stopping problems and an 
effective characterization of the value function is possible. 



6 Appendix 

6.1 Proof of Lemma 13.21 



To make the proof more intuitive, we will work with (I3.13h rather than with (|3.14p where the 
integration part is converted to g functions. For this purpose, it is convenient to define the following 
two operators J\4 ■ T~t — ► T~t and C a : TL — > 7i: 



M. u(x) = sup[K(x, y) + u(y)} 



and 



C u(x) = supE x 

TS5 



I e- as f{X s )ds + e- ar M u(X T 
Jo 



(6.1) 



(6.2) 



Hence we can proceed with the arguments developed in Davis (|1992l ). In terms of the two operators 
just defined, (|3.13j) becomes 



w n +x(x) = supE 2 

rS-S 



/" e- as f(X s )ds + e~ ar MoW n (X T 
Jo 



C w n (x). 



(6.3) 
(6.4) 



(1) w n = v n for all n: Let us now prove v n [x) = w n [x) for all n £ N. We show already v\[x) = w\(x). 
We assume, to make an induction argument, that v n {x) = w n {x) and prove v n+ \{x) = w n+ i(x). 
We should note that, for each n, the optimization problem in (|6.2p is an optimal stopping problem. 
Hence by Proposition 12.31 we can confine the set of strategy S n in (|3.9j) into a smaller set, i.e. 
barrier strategies; 



S n — {y £ S n : Ti, i £ N is an exit time from some interval.}. 



(6.5) 
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Now we proceed with the planned induction argument, 



v n +i(x) = sup E a 



sup E a 

(T,f)eSi 



sup E x 



/■oo 

/ e— /(X s )d S + ^e- aT ^(X Tt _,X Ti 
Jo ™ 



e- QS /(X s )ds + e- QT (^(X T _,X T ) + u n (X T )) 



e- as f(X s )ds + e- QT (^(X r _, X T ) + w n (X T )) 



supE 2 
res 



/ T e - as f(X s )ds + e- aT M oWn (X T 
Jo 



C w n (x) = w n+ i(x) 



for all x E R. The second equality is due to the strong Markov property justified by (|6.5p . The 
third equality is by the induction hypothesis. This proves the first statement of the lemma. 

(2) v(x) = lim n _ +00 w n (x): Since w n is monotone increasing, the limit w{x) = lim n _ > w n (x) 
exists. Since S n C 5, w n {x) < v(x). Hence w{x) < v{x). To show the reverse inequality, we define 
S* be a set of interventions such that 

S* = {v e S : J v {x) < oo for all x G R}. 

Let us assume that v(x) < +00 and consider strategy v* 6 S* and another strategy v n that 
coincides with u* up to and including time T n and then takes no further interventions. 



r\x)-r n {x)=w 

which implies 



e- as (f(X s )-f(X° s ))ds+ £ e-^K(X Ti _,X Ti 

i>n+l 



u v r n (x)\ <E* 



a 



i>n+l 



As n — > +00, the right hand side goes to by the dominated convergence theorem. Hence it is 
shown 

v(x) = sup J u (x) = sup J v {x), 

so that v(x) < w(x). Next, consider the case of v(x) = +00. By Proposition 13.31 we have q = +00 
in this case. Then by the recursive method described in Section [37T] we see that Vi(x) = wi{x) = 00. 
By the first statement of this lemma, we can conclude v n (x) = w n (x) = 00 for all n 6 N, obtaining 
v(x) = limj^oo w n (x). This completes the proof of the second statement. 
(3) w = C w : Since w n | w, we have the following chain of equalities: 

M w(x) = sup[K(x,y) + 117(3/)] = supsup[iT(x, y) + w n (y)] 



supsup[K(x,y) + w n (y)} = su-pM w n (x). 

neN^eR neN 
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In view of this, if we take the limit on the both sides of f|6.3|) as n — ► oo, by the monotone convergence 
theorem, 



w(x) = supE x 



[ T e- as f(X s )ds + e~ aT M w{X T ) 
Jo 



This shows that w = C Q w. Suppose w'(x) satisfies w' = C w' and majorizes g{x) = Vq(x). Then 
w' = C w' > C q vq = w\. If we assume that w' > v n , then 

w' = C a w' > C Q v n = v n+ \ = w n+ i. 

By the induction argument, we have w' > w n for all n, leading to w' > limn^oo w n = w. Thus it 
shows that w is the smallest solution majorizing g of the functional equation, w — g = C(w — g). 
This completes the third statement of the lemma. 
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